Variable-constraint classification and quantification of radiology reports under the ACR Index

We apply hierarchical supervised learning technology to the problem of assigning codes from the wellknown ACR Index (a ‘‘double-hierarchy’’ classification scheme from the American College of Radiology) to radiology reports. This task is actually two classification tasks in one: the former uses a first hierarchy of codes describing anatomic locations, and the latter uses a second hierarchy of codes describing pathologies, where the two hierarchies are closely intertwined. A requirement of each such classification task is that the document be placed in exactly one node of depth P2 of the ‘‘anatomic location’’ hierarchy and in exactly one node of depth P3 of the ‘‘pathology’’ hierarchy; this makes our task a (fairly uncommon) variable-constraint classification task, since at the first levels of the hierarchy (2 for anatomic location, 3 for pathology) we need to use a standard ‘‘exactly 1 class per document’’ constraint, while at the lower levels we need to use an ‘‘at most 1 class per document’’ constraint. We have used a large dataset of about 250,000 radiology reports written in Italian and an adaptation of our TREEBOOST.MH learning algorithm to variable-constraint classification. Notwithstanding the extreme difficulty of the task (given by the fact that the two codes had to be picked out of a pool of 719 codes for anatomic location and 5269 codes for pathology, respectively) our system displayed good accuracy, indicating that it may represent a viable tool for semi-automated classification of medical reports. We also analyzed the quantification accuracy of our system (i.e., the ability of the system at correctly estimating the frequency of the individual codes), a concern of special interest in epidemiology; the results show that our system has excellent quantification accuracy, making this system a valuable tool for the fully automated coding of radiology reports for epidemiological purposes. 2012 Elsevier Ltd. All rights reserved.